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(57) Abstract: A portable hand-held device and method is presented for the kinematic control of hand-held, devices, including the 
control of their function, and the control of the information presented in the display of a portable device. The control is obtained by 
moving the entire hand-held device. The movement or the position of the device is sensed e.g. by accelerometers or a small camera 
mounted on the device. Specific movements are then used e.g. to turn the device on and off, to scroll the information presented in a 
device's display, or to zoom the image in or out. 
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APPARATUS AND METHOD FOR THE KINEMATIC CONTROL OF HAND- 
HELD DEVICES 
BACKGROUND OF THE INVENTION 
1. Field of the Invention 
5 The invention is directed to hand-held portable 

devices comprising at least one movement measuring means 
capable of controlling one or more functions of the 
device as well as controlling the presentation of 
information in the display of the device. The movement 

10 measure means sense the movement of the device or its 
position relative to another object. The invention also 
relates to a method for the kinematic control of a 
portable hand-held device by moving the device and 
measuring the movement of the device or its position 

15 relative to another object by movement measurement means. 
Advantageously , the invention provides a hand-held 
portable device and method for conveniently controlling 
the function of the hand-held device and the presentation 
of information in the display of the device. 

20 2. Description of the Prior Art 

Hand-held portable devices such as cellular phones , 
cellular radios, personal communicators, beepers, GPS 
units and others, are decreasing in size and becoming 
more versatile in function. For example, cellular 

25 telephones and personal communicators are used to 
transmit and receive data, documents, text, and pictures, 
to connect to the Internet, for e -commerce and other 
applications. At the same time, their small physical 
dimensions make it difficult to control their multiple 

30 functions using traditional approaches such as a 
keyboard. As used herein, the term "keyboard" refers 
here to any arrangement of buttons that can be pushed, 
joystick, touchpad and the like. This problem becomes 
more severe as the device dimensions shrink, for example 

35 when a cellular phone becomes as small as a wristwatch. 
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These devices usually have a small display , and it is 
often desired to use this display to present large 
amounts of information such as pictures or documents . 

The traditional approach to the control of hand-held 
5 devices and the information presentation in their 
displays is to use specialized keys, e.g., for turning 
the device and various functions on or off, for selecting 
items from a menu, or for scrolling the information 
presented in the display. The traditional approach has 

10 several limitations: in a small device there may not be 
sufficient space for conveniently locating multiple keys, 
and the use of multiple keys may be confusing and 
difficult to remember. 

It is therefore the object of the present invention 

15 to provide improved hand-held portable devices and a 
method for conveniently controlling the function of such 
devices and the presentation of information in their 
display. The new method improves upon existing methods 
by reducing the number of keys required to control the 

20 device and the information presented in its display, and 
by making the control simpler and easier to learn and to 
remember . 

SUMMARY OF THE INVENTION 

In accordance with the present invention, movement 

25 of a portable hand-held device is used to control various 
functions of the device and the information presented in 
its display. The term "hand-held" is used here to refer 
to any portable device that may be moved, translated or 
rotated, by the user during the operation of the device, 

30 including, for example, a device that may be worn as a 
wristwatch rather than held in hand during its operation. 
The invention also relates to hand-held remote control 
units that are used to control the function of other 
devices that are not by themselves portable, such as a TV 

35 set, video player or a DVD unit. 
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In accordance with the Invention, the device is equipped 
with elements that measure the movement of the entire 
device (e.g. accelerometers) or its position relative to 
other objects (e.g. with a small camera mounted on the 
5 device) . Specific movements are used, in addition to the 
possible use of control keys, to control different 
functions of the device. For example, lifting a cellular 
phone will activate it to respond to a calling signal, 
putting it back down will terminate the call . A small 
10 shaking movement of the device will turn it on if it is 
in the off state, and turn it off if it is in the on 
state. 

Another example of the many advantages of the 
invention is selection from a menu. In the traditional 

15 approach, a menu presents a number of alternative 
functions to choose from, and the selection of an option 
from the menu is performed by operating a keyboard, for 
example by pressing buttons , or moving a cursor to a 
desired position in the display. According to the 

20 present invention, the selection of items from a menu can 
be obtained by moving the entire portable device in a 
particular direction. For example, two options to select 
from may be presented in the display, one on the left 
side of the display and the second on the right side. A 

25 movement of the device to the left will select the first 
option, and movement to the right will select the second 
option . 

Movements can also be used to control the 
presentation of information in the display. For example, 

30 the display may show a small portion of a document. The 
document may contain text, pictures, graphics and will 
typically be too large to be presented all at once within 
the display of the hand-held device. Movements of the 
device in space will change the displayed information to 

35 show different portions of the document. If the device 
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is moved to the right, for example, then the displayed 
information will be changed accordingly, showing a new 
portion of the document to the right of the previously 
presented location. Moving the device closer or further 
5 from the user, or in a perpendicular direction to the 
display, will zoom the image in and out, changing the 
resolution of the presented image or the size of the 
displayed text or picture. 



BNSDOCID: <WO 0186920A2_I_> 



WO 01/86920 PCT/IB01/00851 

- 5 - 

BRIEF DESCIRPTION OF THE DRAWINGS 

Fox a better understanding of the present invention, 
reference will now be made, by way of example, to the 
accompanying drawings, in which: 
5 Figure 1 shows a schematic diagram of a hand-held 

device in accordance with the invention. 

Figures 2 to 4 illustrate the kinematic control of a 
hand-held device in accordance with the 
invention . 

10 Figure 5 illustrates an enabling means to avoid 

accidental activation of the hand-held device. 

Figure 6 shows a schematic block diagram 
illustrating an embodiment of practicing the invention . 
DESCRIPTION OF THE INVENTION 

15 The hand-held device of the invention is equipped 

with elements for measuring the movements of the entire 
device , e.g. by three accelerometers that measure the 
acceleration along three independent directions . 
Alternative, or in addition, the position of the device 

20 relative to other objects can be measured, e.g. by using 
a small camera mounted on the device. Such a camera can 
take a sequence of pictures of static nearby objects, for 
example, a tabletop and use these pictures to determine 
the position or the movement of the camera with respect 

25 to such objects. The measured movements of the device 
are used for two purposes . One is for controlling 
different functions of the device using motion commands 
instead of, or in combination with, the operation of 
other input devices such as a keyboard. The second 

30 purpose is to control the information presented in the 
display of the device. 

Additional mechanisms can be added to avoid unwanted 
changes in the information presented in the display, or 
unwanted activation of control functions, by small 

35 movements of the device, caused for example by small 
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instabilities of the hand holding the device. For 
example, it can be arranged that when the movements are 
small in terms of overall change in position, or when the 
speed or acceleration of the movement are below critical 
5 pre-defined values (that correspond to small unintended 
movement) , such movements will be ignored and will not be 
used to change the information presented in the display 
or to activate control functions of the device.. 
These functions are further explained with reference to 

10 the drawings . 

Figure. 1 shows a schematic diagram of a hand-held 
device such as a cellular telephone, and the notation 
used for describing directions of motion. The x and y 
directions are in the plane of the device and its 

15 display, and the z direction is perpendicular to this 
plane. Since the display is usually facing the user, the 
z direction is normally in a direction that brings the 
device closer to, or further away from, the user. 

Figures 2, 3, 4 illustrate the use of the invention 

20 to control the presentation of information in the display 
of a hand-held device. Figure 2 is a schematic 

representation of a document 201 and a virtual window 202 
which is a part of the document selected by the algorithm 
that controls the presentation of information in the 

25 device's display. This selected part is shown in the 
Figure as an outlined window placed over the document and 
selecting a part thereof. The document may contain text, 
graphics, still or moving pictures, and any combination 
thereof. The document or a large part of it is typically 

30 stored temporarily in a memory unit of the hand-held 
device. For example, the user may use a cellular phone 
to access information via the Internet. Information from 
the Internet will then be downloaded and stored 
temporarily in the device's memory. In this example the 

35 information contained in the entire document cannot be 
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all presented at once ±n the display of the hand-held 
device. Window 202 contains the portion of the document 
currently presented in the display of the hand-held 
device. The content of the window 202 is currently 
5 presented in the display 204 of the hand-held device 203. 
Figure 3 illustrates in a schematic form how the display 
of information from a document is controlled by moving 
the hand-held device. Device 303 and display 304 are the 
same as device 203 and display 204 in Figure 2 . The 

10 device is now moved to a new position, in this example 
down and to the right in the Figure. Figure 303-A shows 
device 303 in its new position, following the movement, 
and 304-A shows the display in its new position, 
following the movement. This movement of the device is 

15 sensed by the elements that measure the acceleration, 
velocity or position of the device, and it is used to 
issue a command to change the information presented in 
the display 304-A. The change of information in the 
display will depend in general on the direction and 

20 magnitude of the movement of the device from its previous 
position to its new position, including in general both 
translation in space and rotational movements . 

This is shown schematically by the movement of the 
window 302, corresponding to window 202 in Figure 2, to a 

25 new position 302-A over the document 301. The new 
information from the document corresponding to the part 
of the document within the new position of the window 
302-A is now presented in the display 304-A. In this 
manner, physical movement of the band-held device in 

30 space corresponds to movements over the document showing 
at each time a part of the document, as if a real window 
was moved in a related manner over the document. The 
document 301can be scanned and different parts can be 
viewed by moving the hand-held device in space. The 

35 movement of the window over the document 301 can 
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correspond exactly to, be proportional to, or be related 
in other ways to the movement of the hand-held device. 
The rate of change of information in the display can be 
related to the speed or acceleration of the device 
5 itself, so that, for example, the document can be scanned 
faster by increasing the speed of movement of the hand- 
held device itself. 

Figure 4 shows in a schematic form the use of 
movement to zoom in and out of the document. Moving the 

10 device in the z direction, for example in a direction 
closer to the viewer, will cause a zoom- in of the 
information presented in the display. For a text 

document, this will typically mean increasing the size of 
the viewed text, and at the same time decreasing the 

15 portion of the document presented in the display. For a 
picture image, zoom-in will increase the magnification of 
the presented image and will typically decrease the 
portion of the picture shown in the display. Movement in 
the opposite direction, along the z-axis but for example 

20 away from the viewer, will cause a zoom-put of the image. 
For a text image, this will typically mean reducing the 
front size in the displayed text, while showing at the 
same time a larger portion of text in the display. For a 
picture image, this zoom-out will decrease the 

25 magnification of the presented image, usually allowing 
the presentation of additional parts of a larger picture, 
but at a reduced magnification. The rate of change in 
magnification (rate of zoom-in or zoom-out) can be 
related to the speed or acceleration of the movement of 

30 the hand-held device, so that, for example, the rate of 
zoom can be increased by increasing the speed of motion 
of the device itself. 

In Figure 4, a document 401 is shown. A selected part of 
this document shown as window 402 was presented in the 
35 display unit prior to a movement of the device in the 
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direction of the z-axis. The device was then moved along 
the x-axis, for example causing it to move closer to the 
eyes of the person holding the device. Figure 403 
illustrates the device in its new position following the 
5 movement along the z-axis direction, and 404 is the 
display of device 403 . The movement is measured and used 
to issue a command to change the information displayed in 
the display unit 404 of device 403. The information 
presented in the display unit 404 following the last 

10 movement of the device is shown schematically in window 
402-A. This window is smaller than window 402. The 
information contained within this window will be 
displayed in the display 404 at a larger magnification 
than the presentation of this information prior to the 

15 movement of the device, and typically a smaller part of 
the document will be displayed at the increased 
magnification. In this manner, movements of the device 
along the z-axis direction cause zoom in and out of the 
information presented in the display of the hand-held 

20 device. 

To avoid accidental activation of the device by 
movement that is not intended by the user to function as 
a command to the device or its display, the use of the 
movement to control the device or its display may be 

25 conditioned on the activation of an enabling switch. 

This is shown in Figure 5. The hand-held device is 
designated as 501; 502 the display of the device; and 503 
is in this particular example a 3?position switch. 
Figure 504 shows a magnified view of the same 3?position 

30 switch. At one position of the switch 503 (shown also in 
504) marked by N (for Neutral condition) , the use of 
movements to control functions of the device or its 
display unit is disabled, and movements of the device 
will not be used to control the device or the information 

35 presented in its display. The same switch is used in 
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this example also to select between the control of device 
functions and control of its display. In position C (for 
^Control') of switch 503 (shown also in 504), movements 
of the device are used to control different functions 
5 such as turning the device on or off. At this position, 
movements of the device will not be used to control the 
display. In position D (for ^Display' ) of the switch 
503, shown also in 504, movements of the device are used 
to control the presentation of information in the 

10 display, as illustrated by examples in Figures 3 and 4. 
Switch 503 can be spring? loaded in such a way, that its 
natural position, in the lack of any external force, is 
the neutral position N. Positions C and D are achieved 
by pressing the switch against the spring* 

15 Figure 6 shows a schematic block diagram of the 

apparatus described. In general, the apparatus contains 
elements for measuring the acceleration, velocity, or 
position of the hand?held device. In the example shown, 
the movement measurements are obtained by three 

20 accelerometers shown as 601?A, 601?B, 601?C in the 
drawing. The three accelerometers measure the 

acceleration of the device along three independent 
directions . Additional accelerometers can be used, for 
example for precise measurements of rotational movements 

25 of the device. The acceleration measurements are fed 
into a processor 602 in the drawing, that can be a 
special processor, or a processor such as a 
microprocessor that is used for other purposes in the 
hand?held device. The processor performs time 

30 integration of the acceleration measurements to obtain 
velocity measurements of the device, and also time 
integration of the velocity measurements to obtain 
measurements of the position of the device at any desired 
time. These integration functions can alternatively be 

35 performed by a dedicated processor connected to the 
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accelerometers rather then processor 602 . 

The processor 602 is connected to a display control 
unit 603 that controls the information presented in the 
display 604. The display control unit 603 is also 
5 connected to display buffer 605 that contains at any 
given time the information selected to be displayed in 
the display 604. The memory unit 606 is used to store a 
larger image, such as a full document or a part of it, 
and a selected part of the document stored in memory unit 

10 606 is transferred to buffer 605 to be displayed in the 
display 604. The same memory 606 is also used to store 
recent movements of the device. This information can be 
used by the processor 602 to allow smooth update of the 
display over time, or for ignoring small movements of the 

15 device. The same memory unit 606 can also store other 
information such as recent commands issued by the user of 
the device. 

The processor receives input also from control keys 
shown together as keyboard 607, that may include a 

20 control switch such as shown in Figure 5, switch 503, as 
well as other keys, control buttons, joystick and the 
like. Processor 602 uses the movements of the device 
together with information from control keyboard 607 to 
issue command signals to the display control unit 603, 

25 and to issue command signals to the device control unit 
608, for example, for turning the device itself or 
different functions of the device on or off. 

It is to be understood that the present invention is 
not limited to the embodiments described above, but 

30 encompass any and all embodiments within the scope of the 
following claims . 
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CLAIMS 

1. A hand-held portable device comprising at least one 
movement measuring means capable of controlling one or 
more functions of the device. 
5 2 . A hand-held portable device comprising a display 
for information and at least one movement measuring means 
capable of controlling the information presented in the 
display of the device. 

3. The hand-held portable device according to claim 1 
10 or 2, wherein the movement measuring means senses the 

movement of the device or its position relative to 
another object. 

4 . The hand-held portable device according to claim 1 
or 2 , wherein the movement measuring means is an 

15 accelerometer . 

5 . The hand-held portable device according to claim 1 
or 2, wherein the movement measuring means is a camera. 

6. The hand-held portable device according to claim 1 
or 2 , wherein the movement measuring means is selected 

20 from the group consisting of an accelerometer, a camera 
and combinations of both. 

7 . The hand-held portable device according to claim 1 
or 2, wherein the device is a remote control unit. 

8. The hand-held portable device according to claim 1 
25 or 2 further comprising a keyboard capable of controlling 

one or more functions of the device in combination with 
the measuring means . 

9. The hand-held portable device according to claim 1 
or 2 further comprising an enabling means for avoiding 

30 unwanted activation of the device. 

10. A hand-held portable device comprising a display for 
presenting information and at least one movement 
measuring means capable of controlling one or more 
functions of the device and the presentation of 

35 information on the screen . 
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11. The hand-held portable device according to claim 10, 
wherein the movement measuring* means senses the movement 
of the device or its position relative to another object. 

12. The hand-held portable device according to claim 10, 
5 wherein the movement measuring means is an accelerometer . 

13. The hand-held portable device according to claim 10, 
wherein the movement measuring means is a camera. 

14. The hand-held portable device according to claim 10, 
wherein the movement measuring means is selected from the 

10 group consisting of an accelerometer, a camera and 
combinations of both. 

15. The hand-held portable device according to claim 10, 
wherein the device is a remote control unit. 

16. The hand-held portable device according to claim 10 
15 further comprising a keyboard capable of controlling one 

or more functions of the device in combination with the 
measuring means . 

17 . The hand-held portable device according to claim 10 
further comprising an enabling means for avoiding 

20 % unwanted activation of the device. 

18. A method for the kinematic control of a portable 
hand-held device comprising moving the hand-held device 
and measuring the movement of the device or its position 
relative to another object by one or more movement 

25 measuring means capable of controlling one or more 
functions of the device. 

19. A method for the kinematic control of a portable 
hand-held device having a display for information 
comprising moving the hand-held device and measuring the 

30 movement of the device or its position relative to 
another object by one or more movement measuring means 
capable of con-trolling the presentation of information on 
the display. 

20. A method for the kinematic control of a .portable 
35 hand-held device having a display for information 
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comprising moving the hand-held device and measuring the 
movement of the device or its position relative to 
another object by one or more movement measuring means 
capable of controlling one or more functions of the 
5 device and the presentation of information on the screen. 
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mounted on the device. Specific movements are then used e.g. to turn the device on and oil", to scroll the information presented in a 
device* s display, or to zoom the image in or out. 
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